Artificial intelligence (AI) is often touted as the most exciting technology of our age, promising to transform our economies, lives, and capabilities. Some even see AI as making steady progress towards the development of'intelligence machines' that will soon surpass human skills in most areas. AI has indeed made rapid advances over the last decade or so, especially owing to the application of modern statistical and machine learning techniques to huge unstructured data sets. It has already influenced almost all industries: AI algorithms are now used by all online platforms and in industries that range from manufacturing to health, finance, wholesale, and retail. Government agencies have also started relying on AI, particularly in the criminal justice system and in customs and immigration control.

Data plays a critical role in machine learning. Every machine learning model is trained and evaluated using data, quite often in the form of static datasets. The characteristics of these datasets fundamentally influence a model's behavior: a model is unlikely to perform well in the wild if its deployment context does not match its training or evaluation datasets, or if these datasets reflect unwanted societal biases. Mismatches like this can have especially severe consequences when machine learning models are used in high-stakes domains, such as criminal justice,1,13,24 hiring,19 critical infrastructure,11,21 and finance.18 Even in other domains, mismatches may lead to loss of revenue or public relations setbacks.

Liu Leqi is a Ph.D. student in the Machine Learning Department at Carnegie Mellon University, Pittsburgh, PA, USA. Her research interests include AI and human-centered problems in machine learning. Dylan Hadfield-Menell is an assistant professor of artificial intelligence and decision-making at the Massachusetts Institute of Technology, Cambridge, MA, USA. His recent work focuses on the risks of (over-) optimizing proxy metrics in AI systems. Zachary C. Lipton is the BP Junior Chair Assistant Professor of Operations Research and Machine Learning at Carnegie Mellon University, Pittsburgh, PA, USA, and a Visiting Scientist at Amazon AI. He directs the Approximately Correct Machine Intelligence (ACMI) lab, whose research spans core machine learning methods, applications to clinical medicine and NLP, and the impact of automation on social systems. He can be found on Twitter (@zacharylipton), GitHub (@zackchase), or his lab's website (acmilab.org).

These creatures are targeting tabletops. One of them will wheel up to a table and ponder for a few seconds to determine if people are seated; if so, it moves on until finding one that's empty. After lingering for a second--maybe taking the algorithmic equivalent of a deep breath before the "Let's do it" moment--the robot twirls and unfurls its limb, stretching the arm over the table to methodically cover the surface with a clear disinfectant. Then it withdraws the arm to squeeze out the excess fluid into a bucket on its base. Task completed, it moves on, seeking another table to swipe.

Apple is stepping up its plans to enter the car market and aims to launch a self-driving electric vehicle in 2025, according to a report. The tech company's much-rumoured automotive project has bolstered its ambitions under new leadership and is pushing for a fully self-driving vehicle with no steering wheel or pedals, said Bloomberg. The car's interior would be designed for hands-off driving, with one possible design featuring passengers sitting around a U-shaped seating formation. Apple's below-the-radar car venture – known as Project Titan – was dealt an apparent blow in September when the executive in charge of its development, Doug Field, defected to Ford. But the iPhone maker appears undaunted by the challenge of entering the competitive electric vehicle market despite a number of senior leadership changes at Titan this year, Field's the most significant among them.

The whirring, squeaking or clicking created by robots' wheels, joints and motors are usually undesirable, and engineers work hard to minimise them. But a research team has found that they can be useful as part of an echolocation system to aid navigation and avoid crashes. Most robots, whether they walk, roll or fly create some sort of background noise. Flying drones, in particular, are extremely noisy.

Figure 1: Our framework classifies language generation tasks into compression, transduction, and creation (left), and unifies the evaluation (middle) of key quality aspects with the common operation of information alignment (right). TL;DR: Evaluating natural language generation (NLG) is hard. Our general framework helps solve the difficulty by unifying the evaluation with a common central operation. Inspired metrics achieve SOTA correlations with human judgments on diverse NLG tasks. Our metrics are available as library on PyPI and GitHub.

One of the central challenges of Natural Language Processing (NLP) systems is to derive essential insights from a wide variety of written materials. Contributing sources for a training dataset for a new NLP algorithm could be as linguistically diverse as Twitter, broadsheet newspapers, and scientific journals, with all the appellant eccentricities unique to each of just those three sources. When an NLP algorithm has to consider material that comes from multiple eras, it typically struggles to reconcile the very different ways that people speak or write across national and sub-national communities, and especially across different periods in history. Yet, using text data (such as historical treatises and venerable scientific works) that straddles epochs is a potentially useful method of generating a historical oversight of a topic, and of formulating statistical timeline reconstructions that predate the adoption and maintenance of metrics for a domain. For example, weather information contributing to climate change predictive AI models was not adequately recorded around the world until 1880, while data-mining of classical texts offers older records of major meteorological events that may be useful in providing pre-Victorian weather data.

Cohere, a startup creating large language models to rival those from OpenAI and AI2Labs, today announced the general availability of its commercial platform for app and service development. Through an API, customers can access models fine-tuned for a range of natural language applications, in some cases at a fraction of the cost of rival offerings. The pandemic has accelerated the world's digital transformation, pushing businesses to become more reliant on software to streamline their processes. As a result, the demand for natural language technology is now higher than ever -- particularly in the enterprise. According to a 2021 survey from John Snow Labs and Gradient Flow, 60% of tech leaders indicated that their natural language processing (NLP) budgets grew by at least 10% compared to 2020, while a third -- 33% -- said that their spending climbed by more than 30%.